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Abstract 



> 

<^^ • In this paper, we present an improved union bound on the Linear Programming (LP) decoding 

in 

QQ ' performance of the binary Hnear codes transmitted over an additive white Gaussian noise channels. The 

bounding technique is based on the second-order of Bonferroni-type inequality in probability theory, 

(^ ■ and it is minimized by Prim's minimum spanning tree algorithm. The bound calculation needs the 

fundamental cone generators of a given parity-check matrix rather than only their weight spectrum, but 

involves relatively low computational complexity. It is targeted to high-density parity-check codes, where 

• 1-^ 

/\^ , the number of their generators is extremely large and these generators are spread densely in the EucUdean 

H ■ 

C^ ■ space. We explore the generator density and make a comparison between different parity-check matrix 

representations. That density effects on the improvement of the proposed bound over the conventional 

LP union bound. The paper also presents a complete pseudo-weight distribution of the fundamental cone 

generators for the BCH[31,21,5] code. 

Index Terms 

Fundamental cone generators. Hunter bound, high-density parity-check (HDPC) code. Linear Pro- 
gramming (LP), LP upper bound, LP union bound, pseudocodewords (PCWs), pseudo-weights, weight 
distribution. 



I. Introduction 

THE calculation of error probability for Linear Programming (LP) decoding of Binary 
Phase-Shift Keying (BPSK) modulated binary codes is often a complex task. This is 
mainly due to the complexity of LP Voronoi or decision regions ^ [|2l- The probability of 
correct decision in an Additive White Gaussian Noise (AWGN) channel, can be obtained by 
integrating a multidimensional Gaussian distribution over the decision region of the transmitted 
codeword (CW). 

LP decoding is a relaxed version of the Maximum-Likelihood (ML) decoding. The codeword 
poly tope [3 1 of ML is replaced by a relaxed polytope, called the fundamental polytope [3|. The 
fundamental polytope arisen from a given parity check matrix. Its vertices are every codeword, but 
it also has some non-codeword. The vertices of the codeword polytope are the all codewords, and 
the vertices of the fundamental polytope are called pseudocodewords (PCWs) Q . The additional 
non-codewords make the decision region fl^ of the LP decoder even more complex than that 
of the ML. Therefore, a derivation of analytical bounds has an important role in evaluating the 
performance of the LP decoder 

The fundamental cone [2J is the conic hull of the fundamental polytope. The LP error proba- 
bility over the fundamental polytope is equal to that over the fundamental cone [|4J. Moreover, it 
is sufficient to consider only the fundamental cone generators flU for evaluating the performance 
of the LP decoder 

The well-known upper bound on the error probability of a digital communication system is 
the Union Bound (UB), which is a first-order Bonferroni-type inequality [|5| in the probability 
theory. The UB of the LP decoder [[D H Q for High-Density Parity-Check (HDPC) codes 
presets inaccurate results due to the high density of fundamental cone generators. In fact, the 
union bound sums all of the pairwise error events as if they were disjoint, but this scenario is 
far from being the case in LP decoding of HDPC codes. 

Each pseudocodeword in the LP decoder can be located in the BPSK signal space Q. What 
the LP decoder does, it chooses the nearest pseudocodeword to the received vector as the most 
likely transmitted pseudocodeword. The ML soft decision decoder has such property as well, but 
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unlike to the LP decoder, its signal space contains only the set of the all codewords. Thus many 
of ML upper bounds can be reused [|8l [|9l [ITOl [011 in the case of LP decoding. 

For a given code, each of its parity-check matrix creates a fundamental cone with different 
pseudo-weight spectrum and geometrical structure, which influences differently on the error 
probability of the LP decoder. Therefore, the geometrical properties of the fundamental cone 
generators are essential to evaluate with a better accuracy the LP decoding error probability. 
Thus ML error probability bounds which use the weight spectrum of the code or those who 
sum the error contribution of each individual codeword become less attractive. In [fTTI a ML 
bound is presented which is based on the second-order upper bound on the probability of a finite 
union of events. And indeed, it uses the geometrical properties of the codewords and considers 
an intersection of pairwise error events, but involves relatively high computational complexity. 

To explore the density of the fundamental cone generators, we have defined the angle graph: 
each generator is considered as a node of a complete undirected graph. The cost of an edge is 
the angle between the generators related to the adjacent nodes. The minimum spanning tree is 
found and its cost distribution is illustrated. Different pattems for various parity-check matrices 
were observed. 

In this paper, we propose an upper bound based on the second-order of Bonferroni-type 
inequality. The bound needs the fundamental cone generators rather than their weight spectrum. 
We call it Improved Linear Programming Union Bound (ILP-UB). It consists of two parts: The 
first term is the LP union bound itself, and the second term is a second-order correction that can be 
optimized by a known minimum spanning tree algorithm. It requires relatively low computational 
complexity since it involves only the Q-function. 

The proposed ILP-UB makes use of an upper bound of the triplet-wise error probability that 
has been introduced earlier in the paper. We derive analytical expression to evaluate the triple- 
wise error probability depending on the angle which they create. And for example, the triple-wise 
error probability for the minimal-weight generators of the BCH[63,57,3] code is calculated. It is 
compared to the triple-wise error upper bound and to the UB in different angles and Signal-to- 
Noise Ratios (SNRs). 
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The proposed ILP-UB was tested on three HDPC codes: Golay[24,12,8], BCH[31,26,3], BCH[63 
,57,3], and on the Low-Density Parity-Check (LDPC) Tanner code [155,64,20] ^^. An improve- 
ment of up to 0.37 dB has been demonstrated over the conventional Linear Programming Union 
Bound (LP-UB). 

This paper is organized as follows. Sec. |II]provides some background on ML and LP decoding. 
The minimum spanning tree problem for undirected graph is also reviewed in Sec. [Ill In Sec. [nl] 
we explore the density of the fundamental cone generators and we check the effect of that density 
on the union bound of the triplet-wise error probability. The problem of finding an LP dominant 
error events is discussed in Sec. |IV] In Sec. |V] we propose an improved linear programming 
error union bound. Sec. |Vl] provides numerical results and discusses some possible direction for 
further research on how to improve the proposed bound. Sec. IV III concludes the paper. 

II. Preliminaries and Definitions 

A. ML and LP Decoding 

In this section we briefly review ML and LP decoding (Tj. Consider a binary linear code C of 
length n, dimension k and code rate R = k/n. Let F2 = {0, 1} denote the finite field with two 
elements. The code C is defined by some mxn parity-check matrix H E F™™ with row vectors 
hi, h2, ..., hm, i.e. C ={x G F2 | xH^ = 0}. The code will be called an [n,k,d] code, in which d 
is its minimum Hamming distance. The code is used for data communication over a memoryless 
binary-input channel with channel law Py|x(y|x). We denote the transmitted codeword by x = 
(xi, ..., Xn), the transmitted signal by x = (xi, ..., x„) and the received signal by y = (yi, ..., y„). 
We assume that every codeword x G C is transmitted with equal probability. Let A denote the 
Log-Likelihood Ratio (LLR) vector with the LLR components A,; = Py|x(l/j|0)/Py|x(yi|l) for 
i = 1, ...,n. The block-wise Maximum Likelihood Decoding (MLD) is 

XA/LD(y) = arg min (x, A) . (1) 



A 

a] 
xec 



Where (x, A) = ^^ XjAj denote the standard inner product of two vectors of equal length. The 
ML decoder error probability is independent of the transmitted CW, therefore, we assume without 
loss of generality that the all-zeros codeword xq is transmitted. Then ||T3]| 
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P!^''^'' {error \ xq) = P. i^wiLoiy) ^ xq | xq) (2) 

= Prl y ||x-y||2 < |||xo-y||2 I Xo > (3) 

y xGC\xo J 

^ P^{ ||x-y||2 < |||xo -ylb I Xo} (4) 



< 



Y.Q[t]^ (« 



x? 



xec\xo 

O I 

2a 

xeC\xo ^ 

Where the Q-function is defined to be Q{x) = -^ /^°° expf — y jdt and ||x||2 = a/X]j 
denote the £2-norm of a vector x. Eq. ([3]) also allows to make a simulation of the error probability 
contributed by a subgroup of codewords. Eq. (|5} is the ML union bound, where dy^ = | |x— xqI I2 = 
2 ^RE}jWh (x) is the Euclidean distance from x to the transmitted signal xq- 

The MLD ([U) can be formulated [3J as the following equivalent optimization problem: 

XMLZ)(y) = arg min (x, A) . (6) 

conv{C) is called the codeword polytope (Si, which is the convex hull of all possible codewords. 
The vertices of the codeword polytope are the all codewords. The number of inequalities needed 
to describe it grows exponentially in the code length. Therefore, solving this linear programming 
problem is not practical for codes with reasonable block length. To make this problem more 
feasible it was suggested [l3J to replace conv{C) by a relaxed polytope V = V{H), called the 
fundamental polytope. 

P = Pi conv(Cj) with Cj = {x G F^ I xhj = O} . 
i=i 

Where conv{C) C conv{Cj) for j = 1, ...,m and hence conv(C) C V{H) C [0, 1]". The number 

of inequalities that describe 'P(-ff) is typically much smaller than those of conv{C). The Linear 

Programming Decoding (LPD) is then 

'^LPD(y) = arg min (w, A) . (7) 

In the case of conv{C) = V{H) the relaxed LP solution equals to that of ML. In the case of 
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conv{C) C V{H) the relaxed LP problem represents a suboptimal decoder which has vertices 
in V{H) which are not in conviC). The vertices of V{H), denoted by V(V{H)), are called LP 
pseudocodewords. 

The fundamental cone ^ ^{H) = /C is defined to be the conic hull of the fundamental 
polytope i.e. the set that consists of all possible conic combinations of all the points in V{H) 
and hence V{H) C K,{H). The LP decoding error probability over the fundamental polytope is 
equal to that over the fundamental cone [|4J. We let M and M_,_ be the set of real numbers and 
the set of non-negative real numbers, respectively. 

Definition 1. ( H, HI) A set ^(/C) = {gi,g2, ...,gM | & G M!^, i = 1,...,M} of M linearly 
independent vectors where ^ = \ X^ctigi | a, G M > are called the generators of the cone /C. D 

It follows from Def . [H that a vector x is in /C if and only if x can be written as a nonnegative 

M 

linear combination of the generators, i.e. x = ^ttjgj where «j G M. Note that a set of generators 

4=1 

is not unique, and that the all-zeros codeword xq ^ Q{K). 

We assume an AWGN channel, where each z'-th transmitted bit perturbed by a white Gaussian 
noise z, with a zero mean and noise power a^ = No/2. The received signal is y = x + z, where 
z designates an n-dimensional Gaussian noise vector with independent components zi, Z2, ■■■, z^. 



We consider a BPSK modulation: the transmitted signal is x = 7 (1 — 2x), where 7 = y/REh in 
which Eb is the information bit energy. The signal-to-noise ratio is defined to be SNR = E^/Nq. 
Following from the above, the LLR vector is A = A ^^^^' y [2], and therefore, the LPD will be 

considered henceforth 

'^LPD = argmin (a;, y). (8) 

Definition 2. (0, O, lIH) Let uj G W^. The AWGN channel pseudo-weight w^^^^^{uj) of 
uj is given by 

where ||x||i = Ylii l^il denote the £i-norm of a vector x. If u; = we define w^'^^^'^{uj) = 0, 
and in the case of a; G {0, 1}" we have w^^'^^'^iuj) = WH^i^)- □ 
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For an easier notation, as we discuss in this paper only AWGN channel, we will use the shorter 
notation Wp{u>) instead of Wp^'^^'^{u>). 

Due to the symmetry property of the fundamental polytope the probability that the LP decoder 
fails is independent of the codeword that was transmitted 0. Therefore, we henceforth assume 
without loss of generality when analyzing LPD error probability, that the all-zeros codeword xq 
is transmitted. 

The set of optimal solutions of a closed convex LP problem always includes at least one vertex 
of the polytope. Therefore, the LPD error probability is 

P^'^'''' {error | Xq) = pJ |J (a;, y) < | Xq I . (10) 

[a;eV(P(H))\xo J 

A pseudocodeword p G V{V) also belongs to the fundamental cone. Thus it can be written as 

M 

a non-negative linear combination of the generators, i.e. p = '^aigi with «j > 0. Therefore, if 

M 

there is p G V{V) such that (p, y) = J^'^i iSi^ y) < 0' then there must be at least one generator 
gjG G{JC) such that (gi,y) < 0. Therefore, the union of the pseudocodewords' error events in 
(flOl ) can be replaced by the union of the generators' error events. 

A vector a; G M" which is not codeword can be located into the signal space in the same way 
as a codeword, i.e ZJ = 7 (1 — 2u). The vector uJvirt — ji^ip'^ was introduced by Vontobel and 
Koetter [|2j|. They showed that the decision hyperplane of u: in the signal space, is at the same 
Euclidean distance from xq and from ujyirt- Note that if a; G C C {0, 1}", then cu^j^t = ^- From 
the above, the LP error probability is then expressed in the signal space as follows. 

P^^^^ {error | xq) = pJ |J \\uJ,irt - ylh < ||xo - ylb | xq I . (11) 

Evaluating the LP error probability by simulating Eq. (ITTI i is not practical, since it involves 
enormous number of generators. However, it allows to make a simulation of the error probability 
contributed by a subgroup of generators. 

Let -Exo^oj = { ll^t^irt — ylb < ||xo — y||2 I xo } denote the LP pairwise error event where 
the received vector y is closer to uJyirt than to the transmitted signal xq- Thus the LP error 
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probability (fTTT ) can be written: 



i'/'^^(error I xo) = PJ |J E^,^^\ , (12) 

and the LP union bound is 

P/'^^(error I xo) < J^ P.jPxo^..}. (13) 

u:eg{IC{H)) 

Let r^^ = ll^"''-*~^oll2 = 7a/wp(c(;) denote the Euclidean distance from xq or from ZJvirt to the 
decision boundary line. Thus the LP pairwise error probability [j2j 

and the LP-UB in Eq. ^3^) can be written as follows [[D Q. 

P/'^^(error I xo) < Yl ^ (v) ' ^^^^ 

^eg(/c(H)) 

B. Undirected Graphs 

In this section, we give a brief overview of some terms from graph theory. By a graph we 
will always mean an undirected graph without loops and multiple edges. We let \V\ denote the 
size of a set V. 

Definition 3. ( ifTTll ) An undirected graph G{V, 8) consists of a set of nodes V and a set of edges 
£. An edge is an unordered pair of nodes {vi,Vj). Associated with each edge {vi,Vj) G £^ is a 

cost c(fi,Wj). n 

Definition 4. ( IfTTll ) A spanning tree of an undirected graph G(V, £), is a subgraph T{V, £') that 

is a tree and connects all the nodes in V. It has \V\ nodes and \S'\ = \V\ — 1 edges, in which 

£' is a subset of £. The cost of a spanning tree T, denoted by cost(T), is the sum of the costs 

of all the edges in the tree. i.e. cost{T) = ^ c{vi,Vj). O 

{vi,vj)eT 

Definition 5. ( ifTTl ) A spanning tree of a graph GiV, 8) is called a Minimum Spanning Tree 
(MST), if its cost is less than or equal to the cost of every other spanning tree T{y, £') of 

G{V,£). n 
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Two popular algorithms for finding an MST in undirected graph are Prim's [fT8l and Kruskal's 
[fT9l . A simple implementation of Prim's algorithm can shows 0(|yp) running time, and both 
can be implemented with complexity of 0(|£^| /o(7|y|). 

III. Generator Density Characterization 

In this section, we explore the density of the fundamental cone generators and we compare it 
to that of ML codewords. As a result, we will later examine how the union bound is affected by 
that density. Let < 6'j, < vr denote the positive angle formed by the vectors cJj and cl>,, which 



is equal to the angle formed by the vectors xof-jj^j^t and xqcJj ^j^ in a BPSK signal space. 

Definition 6. Let uji, 0^2, ..., ct^Af ^ Wl be a set of vectors. Consider each vector as a node of an 
undirected graph G{V, £), with an undirected edge joining each pair of nodes u^i and ujj, denoted 
by (cjj, ujj). An edge (cJj, uJj) E £ has a cost that equal to the angle between the vectors related 
to the adjacent nodes, i.e, c(a;j,CL»j) = 9ij. The graph G{V,£) will be called the angle graph. 
Note that the angle graph is a complete graph; it has \V\ nodes and |V^|(|V^| — l)/2 edges. D 

Definition 7. Let T{y,£') be an MST of the angle graph G{y,£) in Def. [61 The MST angle 
distribution is defined to be the cost distribution of the all edges (ojij, uJj) in the graph T(y, £'). 
For easier notation, we will use the shorter term angle distribution instead. D 

Example 8. Let Hq' [[D and Hq" (fT6l) be parity-check matrices for the extended Golay[24,12,8] 
code. The former matrix was introduced by Halford and Chugg [|20l , the latter is a systematic 
parity-check matrix. Fig. [T] presents the angle distributions of the first 759 minimal-weight 
generators of Hq' and Hq" (generators with equal pseudo-weight were ordered randomly.). For 
a comparison, the angle distribution of the 759 minimal-weigh ML codewords is presented as 
well. The average angle of Hg', Hq" generators and of ML codewords are : 1.43°, 10.69° and 
60°, respectively; and their Standard Deviations (STDs) are: 3.38°, 8.72° and 0°, respectively. 
Note that Hg' and Hg" have two different generator matrices, however, both have the same 
angle distribution for their 759 minimal-weight CWs. It is clear from Fig. \T\ that Hg' generators 
are much crowded than those of Hg", and between these three distributions the ML codewords 
are spread most widely and evenly in the Euclidean space. 
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Fig. 1. Angle distributions for the extended Golay[24, 12, 8] code of the first 759 minimal-weight generators of the parity-check 
matrices Hq' and Hqh , compared to the angle distribution of the 759 minimal-weight ML codewords. 
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Example 9. The error probability contributed by two vectors depends on the angle between them. 



Let a;,-,a;,; G 



i,^j 



be vectors with an equal pseudo-weight, and let ,^1 and ,^2 be the two independent 



Gaussian random variables obtained by projecting the noise vector z onto the plan determined 
by the vectors X-ouJi^mrt and xo^j>irt- We refer to the probability Pr {-Exo-^t^i U-^xo^t^j} ^s the 
triplet-wise error probability, that is cJj or oJj was decoded when the all-zeros codeword was 
transmitted. The triplet-wise error probability depends on the angle 9ij and it can be obtained 
by integrating a two dimensional Gaussian distribution over the darkened regions T^i and 7^.2 
in Fig. |2] ETll . Without loss of generality, we assume that ujj is placed on ,^1 axis, r^^- and r^^ 
denote the Euclidean distances from the decision boundaries lines of Wj and Wj, respectively, to 
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the all-zeros codeword. In the case of vectors of equal pseudo- weight, r^^. = r^^ . The decision 
region boundary lines of u^i and ljj are ^2 = —o-^i + b and ,^1 = r^^ , respectively. The uji 
boundary line crosses ,^2 axis at point b = r^J sinOij and its slope is a = tan(90 — %). The 
intersection between the two boundary lines occurs at point (^1,^2) = ('^t^j' —o-r^. + b). 

There are various numerical integration ways [|22l to evaluate the triplet-wise error probability. 
Another possibility, is to approximate it by sum of Q-functions as follows. 



Pr {^xo^o., U ^xo-..., } = Pr{ni} + P,{7^2} ^ Q (^) + 



fc=0 



Q 



-a(e; + kA^i) + b 



a 



Q 



ei + fcA^i 



a 



Q 



ei + (A; + l)A6 



a 



(17) 



Pr{TZi} is equal to an LP pairwise error probability (IT4l . Pr{TZ2} is calculated as follows. The 
region 7^2 is divided into rectangles of a width A,^i which are parallel to the ,^2 axis, as shown 
in Fig. [2l Each rectangle starts from a point on the decision boundary line of <-Jj and goes to 
infinity in the opposite direction of ,^2 axis. The multiplication inside the sum of Eq. (fTTI) is 
the probability that the noise components ^1 and ^2 are within the k-th rectangle. Since a two 
dimensional Gaussian distribution converges to zero as ,^1 goes to infinity, it will be sufficient to 



sum from /c = to a large k such as 
side of the line ^2 = ^i,max- 



si. max 



where all the rectangles are located on the left 
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Fig. 2. The LP triplet-wise error region in the signal space. 



Example 10. Consider the BCH[63,57,3] code. The fundamental cone of the systematic parity- 
check matrix created by the generator polynomial x^+x+l has 11, 551 minimal- weight generators 
of pseudo-weight three. The angles between them varied from 5.85° to 90°. The triplet-wise error 
probability of its two minimal- weight generators depends on 9ij is presented in Fig. |3l It was 
calculated by Eq. (fTTj ) for and 8 dB SNR in different angles. The triplet-wise union bound 
which is 2Q (^) is presented as well, ^i^max and A^i was chosen to be 2000 and 1/2000, 
respectively. From Fig. |3]one can observe that the lower the SNR and the smaller the angle are, 
the worse is the UB. The figure also presents a triplet- wise error probability upper bound which 
is tighter than the UB and it will be introduced in Sec. |V] 
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Fig. 3. Comparison between the triplet-wise error probability, its union bound and the upper bound in different angles of two 
minimal-weight generators of BCH[63,57,3], when the all-zeros word was transmitted. 
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IV. The Problem of Locating Dominant Error Events of LPD 



Consider a ML decoding of a binary-linear code BPSK-modulated over an AWGN channel. 
The decoder performance can be evaluated by considering the contributions of the most dominant 
error events to the probability of error. That dominant error events, especially in the higher SNR 
region, are the minimal weight codewords. 

In this section, we will examine whether the minimal- weight generators of LP decoding have 
such a property as well. We let wh(x) denote the Hamming weight of x, which is the number of 
non-zero positions of x. Let U7™"(C) denote the minimum Hamming weight of a linear code C, 
and let w™"(/J) denote the minimum AWGN channel pseudo- weight of a linear code defined 
by the parity-check matrix H. We will use the shorter notations w^*" and wT'^"' in case where 
the discussed code and matrix are mentioned explicitly. We let /CgubC /C denote a sub-cone of 
the fundamental cone which created by a chosen subgroup of generators. The LPD(/Csub) Frame 
Error Rate (FER) can be obtained by simulating Eq. (fTTT ). In the next example, we will study 
the error probability contributed by a subgroup of codewords and generators for the extended 
Golay[24,12,8] code. 

Example 11. The extended Golay[24,12,8] code has a total 4,096 codewords of which 759 have 
minimal Hamming weight of ty^*" = 8. The fundamental cone of the parity check-matrix Hg' 
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has a total of 231, 146, 333 generators of which two have minimal- weight of ty"^*" = 3.6 [[Tj. 
Simulating the error probability by Eq. (|3) shows that the minimal- weight CWs describe well the 
MLD performance at the whole range of SNR, which is not the case for the first 759 minimal- 
weight generators for LPD. For instance, consider the error rate of 10"^, it was found that the 
difference between LPD(/Csub) and LPD(/C) is about 2.5 dB. The angle distributions which were 
presented in Fig. \T\ support this result: the average angle of that group of generators is as small 
as 1.43°, and the average angle of the ML minimal-weight CWs is 60°. D 

There are number of reasons why the minimal-weight generators are often not a dominant 
subgroup of LPD: (a) There is no guarantee for significant number of generators with minimal 
pseudo-weight. The fundamental cone of Hg' for example, has only two. (b) A subgroup of 
generators can be very crowded, which significantly reduces their contribution to the error 
probability, (c) Unlike MLD which has distinct subgroup of minimal-weight codewords, LPD 
often has a continuous-like weight distribution. For example, the BCH[3 1,21,5] code of parity- 
check matrix Hbch^^^^i] CSI) has 627,052,479 generators. The pseudo-weight distribution of 
these generators is presented in Fig. |4] Its smooth distribution makes it difficult to locate a 
minimal- weight dominant subgroup. 

In LPD, a potential subgroup to be a dominant is taking all generators of weight Wp < w^*". 
This group is not empty since u'™*" < w^*" E3]| . however, it may contains enormous number 
of generators. For example, Golay[24,12,8] has only 759 minimal- weight CWs of w^™ = 8, 
but the fundamental cone of parity-check matrix Hq" has 143, 757, 418 generators of weight 
Wp < u;;^*" = 8. 
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Fig. 4. A complete generators' pseudo-weight distrihution for the BCH[31,21,5] code of Hbch,^^ 211 ^^^ 627,052,479 
generators. 



V. Improved LP Union Bound 

In this section, we propose an improved union bound for LP decoding of a binary linear code 
transmitted over a binary-input AWGN channel. This bound is based on the second-order of 
Bonferroni-type inequality in probability theory [5], also referred to as Hunter bound [l24l . For 
any set of events Ei, E2, ..., Em and their complementary events, denoted by E^, E2, ..., Elj, 



M 



M 



Pr[[JEA=J2Pr[E^r\ 



\j=l 



i=l 



i-1 



(19) 



Let denote the M! possible permutations of the indices of the error events Ei, E2, ■■■, Em by 
n(l,2,...,M) = {7ri,7r2,...,7rM}. For a given IT, let A = {112^3, --Am} denote the (M^ - M)/2 
possible sets of indices in which ttj G {tti, 112, ..., 7rj„i} for i = 2, 3, ..., M. Hunter [|24l presented 
the second-order bound of Eq. (fT9] ) as follows. 

/ M \ M M 

Pr U^O - 5Z^'-(^-') - Y.^r{E.^ ^ ^-J- (20) 



\i=l 



i=l 



1=2 



Minimization of the Right-Hand Side (RHS) of Eq. ( f20|) is required to achieve the tightest 
second-order bound. Using the sets of the indices A and IT, the minimization problem can be 
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written as follows Ml [|24l . 

[JE, < Y,Pr{E.) + ™ \ -Y.^r{E.. n ^^J I . (21) 

The first sum goes through over all the indices 1 to M of the error events, thus E.,,- could be 
changed to Ei. 

Consider each of the random events E^ as a node of an undirected graph G and the intersection 
{Ei n Ej) as an undirected edge joining the nodes Ei and Ej, denoted by {i,j), with a cost 
c{i,j) = Pr{Ei n Ej). Hunter [|24l showed that a set of (M — 1) intersections may be used in 
the second term of Eq. (|2TI) if and only if it forms a spanning tree of the nodes {Ei}^^^. Thus 
the minimization problem of Eq. (1211 ) can be written equivalently [|24ll . [fTOll . 

Pr ( U^H ^ J^^riEi) + mm I - ^ Pr{Ei n E,) i . (22) 

Where r is a spanning tree of the graph G. The problem is to find a graph r which minimizes 
Eq. (I22l) over all possible spanning trees. The solution for that is known as the solution of the 
minimum spanning tree problem and has been proposed by Prim [jTSlI and Kruskal [il9ll . 

Consider the event Ei as the pairwise error event i?xo->wr I^i order to upper bound the LP 
decoding error probability in Eq. ([T2T i by the second-order upper bound (|22l) . the probability 
Pr {E:x.o^uji ri-^xo^ajj} is required, or instead, its lower bound. The probability of intersection 
of two events can be expressed using the inclusion-exclusion principle in probability theory, 

Pr I E^o^i^i \ I -^xo^tjj I = -Pr {-E'xo^aJi} + Pr {-Exo^oj^ } ~ -^r | -^xo^oJi [J -E'xo^ojj | • (23) 

The first and the second terms in the RHS of Eq. (l23l) are the LP pairwise error probability ([14}, 
the third term can be upper bounded by the following theorem. 

Theorem 12. Let u)i, ujj G M" be vectors of a pseudo-weight Wp{uJi) ^ Wp{ujj). The LP triplet- 
wise error probability 



25 th Febraary 2013 DRAFT 



17 



Q — + TT^e ^ 



Pr{E^o-^...[JE^,^^^}<rmn\ V ^ J 2vr ' . (24) 



«ivj+'3iv;-'3if^j'3iv 



Proof: Let ^ = ^^ + ^2 be a random variable with Chi-square distribution [|25l with two 
degrees of freedom, i.e. 

/(O = ^e-5^f/(0, (25) 

in which U{-) is the unit step function. Without loss of generality we assume that Wp{uJi) < 
Wp{u)j). With the help of Fig. [5] the triplet- wise error probability, 

Pr{^xo^c..U^-0^-.} ^ Pr[\JnA<Y,Pr{ni] (26) 

Pr(7^l) + P^(7^2) Pr{Tl-i) Pr{Tli) 

= Q(— )+^e-S^. (28) 



From the noise symmetry, each of the probabilities Pr (TZi) or Pr{Tl2) equal to |<5 (-^ 

PriTZs) is the probability that of ^f + ^| lies in the region outside a circle of a radios r^^ created 
by the central angle %. Pr [^ > r^^ ) was calculated in Eq. (ITTI) by integrating the Chi-square 
distribution (|25]) from r^ to oo. Thus for two vectors of pseudo-weight Wp{uj.i) ^ Wp{ujj) 



rmnir^.^ruj,,] \ f,o- "'°"'^''" i'''"j^ 



The triplet-wise error probability can also be bounded using the inclusion-exclusion principle 
as follows. 

Pr {Bx.,^^, [J Bx.,^^, } = Pr {Bx,,^^,} + P {Bx.^„, } - P- {e^.^^, f] E^.^^. } (30) 



2 «iv^«iv;-«ivj'3iv;' <^" 

The transition from Eq. (|30] ) to Eq. (1311 was done by lower bounding P^ {-E'xo^oji fl -^xo^ojj } at 
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its lowest value Q ( -^ ) Q ( -^ ) accepted in Oij = 90°. Finally, selecting the minimum between 
Eq. (|29l) and Eq. (|3T]) completes the proof. ■ 




Fig. 5. The region in the signal space used to bound the LP triplet-wise error probability (uip(aJi) 7^ Wp{u:j)). 

Example 13. We continue Ex. \T0\ The triplet- wise error probability upper bound of Theorem 
was calculated for two minimal-weight generators of the BCH[63,57,3] code. It is presented in 
Fig. [3] together with the previous results of Ex. (TO] We can see that the smaller the angle and lower 
the SNR, the more improvement the triplet- wise error upper bound has over the union bound. 



Note that because — oc ^ySKR^w^(ijJ), changing the pseudo-weight of the generators will have 
the same effect as changing the SNR. Thus this bound is expected to have more improvement 
on low pseudo-weight generators. D 

In the next theorem, we propose an improved UB for the LP decoding. 

Theorem 14. Let Q{IC{H)) be a set of cone generators of a parity-check matrix H. For each 
uJi E Q the pairwise error event i^xo-^oji is considered as a node of a complete graph GiV, £). Let 
(cjj, ujj) denote an undirected edge joining the nodes related to the events -Exo^w, (^nd -Exo^ojj- 
riy^E') is denoted for a spanning tree of G{y,£). The LP decoding error probability can be 
upper-bounded by 
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/',^^^(error|xo)< J] Q (-) 



t^eg(/c(i/)) 



^ max(r^,,r^J \ ^ % ^ 



* J 



+ min \ 2, ™^ < V ^ / ^^ } \0'2) 



cr / V cr 



We call this bound the Improved LP Union Bound (ILP-UB). The first term is the LP union 
bound itself (fT5l) . the second term is a second-order correction. 

Proof: To prove this, we will apply Hunter bound for the LP error probability. First, we find a 
lower bound for P^ {-Exo-s>t^i fl -^xo-s>t^j } • by substituting the upper bound of Pr {-Exo^u^i U -^xo^u^j } 
(l24l) into the inclusion-exclusion principal (1231) . we will have 

Pr {-Exo^o;, n -Sxo^o;, } > 






maa;(ru,i,r^,)\ % _ 



'^i ; " -2^- 






2(T 



max <( V cr / ^TT ^ _ (34) 

« (v) « (v) 

Applying Hunter bound (|22)) for LP decoding error probability ([T2)) and substituting into it the 
expression in (l34l) together with the LP pairwise error probability (fT4l) . will give the desired 
result. ■ 

Given a set of generators ^, the running time of ILP-UB is equal to that of finding an MST on 
a complete graph G{V,S). It can be obtained by Prim's algorithm with a complexity of 0(|^p). 
The LP-UB for a comparison, for a given set of generators has running time of 0(|^|). 
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VI. Results and Discussion 

In this section, we provide results to show the improvement of ILP-UB over LP-UB. For this 
purpose, we examine four codes, three HDPC codes: extended Golay[24,12,8], BCH[31,26,3], 
BCH[63,57,3]; and one LDPC Tanner code [155,64,20] lfT2ll . The parity-check matrices we use 
for Golay[24,12,8] and BCH[3 1,26,3] are H'^ ^ and Hbchi3,,26] <E3' respectively; and for 
the BCH[63,57,3] we use a systematic parity-check matrix created by the generator polyno- 
mial x^ + X + I. The minimal pseudo-weight of the extended Golay[24,12,8] is w™" = 3.2. 
BCH[31,26,3] and BCH[63,57,3] have the same minimal pseudo-weight: i^™" = 3; and the 
Tanner code [155,64,20] has w™" ^ 16.403 [HI. 

Because of the enormous number of cone generators, we chose representative subgroups: 
for the BCH[31,26,3], BCH[63,57,3] and Tanner code [155,64,20] we chose all the minimal- 
weight generators that are 1,185 , 11,551 and 465 generators, respectively. Because the extended 
Golay[24,12,8] code has only 165 minimal- weight generators we chose for it the first 231 
generators of a weight equal or less than Wp = 3.25. 



HbCH[s,^26] 



/lOOOOlOOlOllOOlllllOOOllOlllOlON 
0100001001011001111100011011101 
0010010110011111000110111010100 
0001001011001111100011011101010 

VOOOOIOOIOIIOOIII 110001101110101/ 



(35) 



Fig. [6] presents the angle distributions according to Def. |7] for the aforementioned codes: 
extended Golay[24,12,8], BCH[31,26,3] and BCH[63,57,3]. Their average angles are 19.85°, 
29.58°, 21.87°, respectively; and their STDs are 13.44°, 13.94°, 13.84°, respectively. 
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Fig. 6. Angle distributions. 

Fig. |7] presents results of the: ILP-UB(/Csub)5 LP-UB(/Csub) and LPD(/Csub) for the chosen 
subgroups of generators. It presents the LPD FER as well. The ILP-UB optimized by Prim's 
algorithm. The ILP-UB presents an improvement over the LP-UB. For instance, we consider 
the error rate of 10"^. For the extended Golay[24,12,8], the difference between LP-UB(/Csub) 
and LPD(/Csub) is about 0.9 dB while ILP-UB(/Csub) shows an improvement of 0.37 dB over 
LP-UB(/Csub). For BCH[31,26,3], the difference between LP-UB(/Csub) and LPD(/Csub) is about 
0.47 dB while ILP-UB(;Csub) shows an improvement of 0.13 dB. And for BCH[63,57,3], the 
difference between LP-UB(/Csub) and LPD(/Csub) is about 0.62 dB while ILP-UB(/Csub) shows 
an improvement of 0.16 dB. 

The results of the LDPC Tanner code were omitted, since the improvement of the ILP- 
UB (/Csub) over the LP-UB (/Cgub) at error rate of 10"^ is dropped to about 0.05 dB. The reason for 
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that is twofold. First, the Tanner code has a large average angle: 35.16°. Second, the generators 
have an high pseudo-weight: if™*" p^ 16.403. These two values are high as compared to the other 
tested codes. 




-■ LPD 

-A — LPD(x;sub) 
-♦ — LP-UB(Kl«b) 
-H ILP-UB(k;*) 




-■ LPD 

-*— LPD(/&ut) 

--»— LP-UB(K.*) 

—I ILP-UB(te.ub) 



Eb/NO [dBl 



5 6 

Eb/NO [dBl 



(a) Golay[24,12,8] code: results for 231 generators with Wp < (b) BCH[31,26,3] code: results of all the 1, 185 minimal- 
3.25 (w™™ = 3.2). weight generators (m™'" = wg™ = 3). 
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(c) BCH[63,57,3] code: results of all the 11,551 minimal- 
weight generators (w^'" = wg™ = 3). 

Fig. 7. A comparison between ILP-UB, LP-UB, LPD and LPD PER for HDPC codes. 

Fig . |7] together with Fig. |6] show that the lower the average angle is, the more improvement the 
ILP-UB has. A small average angle is typical for HDPC codes, therefore, the advantage of ILP- 
UB over the LP-UB will be reflected better on such type of codes. But on the other hand, as the 
larger the average angle is, the better the LP-UB will be. Fig. |7a] presents the highest improvement 
of the ILP-UB (/Csub) among the other codes. This result correlates to Golay's smallest average 
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angle: 19.85°. However, it presents the largest gap to its LPD(/Csub)- This apparently happens 
because there are a significant probabilities of intersections between three error events or more. 
Bukszar and Prekopa have suggested [26] a third order upper bound on the probability of a 
finite union of events. Their bound considers intersections of two and three events. They proved 
that this third order bound, which is obtained by the use of a type of graph called cherry tree, is 
at least as strong as the second-order bound. Therefore, implementing such a bound will improve 
(or at least will be equal to) the proposed ILP-UB. 

VII. Conclusions 

In this paper, we have presented an improved union bound on the error probability of LP 
decoding of binary linear HDPC codes transmitted over a binary-input AWGN channel. It is 
based on the second-order upper bound on the probability of a finite union of events. It has 
low computational complexity since it only involves the Q-function. It can be implemented 
with running time of 0(|^p), where ^ is a set of generators of the fundamental cone arisen 
from a given parity check matrix. We examined the proposed bound for several HDPC codes: 
Golay[24,12,8], BCH[31,26,3], BCH[63,57,3], and for the LDPC Tanner [155,64,20] code. The 
improvement of the proposed bound over the union bound presents dependency on the pseudo- 
weight of the generators and their density. We studied and compared the generator density through 
the angle distribution of various codes and parity-check matrices. Finally, a third order upper 
bound was proposed, it is based on a type of graph called cherry tree, and is left open for further 
research. 
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